Painting image sentiment prediction is a research hotspot in affective computing. At present, there are few sources of abstract paintings and a small sample size; most of its sentiment analysis uses low-level features of the image, and the accuracy is not high. To resolve these problems, a sentiment prediction of small sample abstract painting image based on feature fusion was proposed. First, the relationship between the basic elements of abstract painting (point, line, plane and color) and human emotions in abstract art theory was analyzed, and according to these theories, the low-level features of abstract painting image were quantified. Second, the transfer learning algorithm was adopted to obtain the parameters from large sample data in the pre-training network, and these parameters were transferred to the target model, and then the target model was fine-tuned on the small sample data to obtain the high-level features of the image. Finally, the low-level and high-level features were fused linearly, and the multi-class Support Vector Machine (SVM) was used to achieve the sentiment prediction of abstract painting image. The experiments were carried out on three small sample abstract painting datasets, and the proposed method was compared with the methods of directly using low-level features. The results show that the classification accuracy of the proposed algorithm is improved, confirming its effectiveness in sentiment research of small sample abstract painting.